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Abstract 


Multimodal biometric systems which fuse information from a 
number of biometrics are most spreads lately because they are 
able to overcome problems in unimodal biometric systems. 
Most of the proposed multibiometric systems offer one level 
of fusion. In this paper, a comparison between two levels of 
fusion has been proposed; a proposed fusion system of three 
biometrics at the feature level based on Particle Swarm 
Optimization approach (PSO) is presented with a new multi 
objective fitness function for PSO has been used. Also the 
score level fusion rule is optimized using (PSO) Particle 
Swarm Optimization. Results shown that matching score 
fusion outperforms matching score fusion in one multimodal 
system (palmprint_Knuckle), while matching score fusion 
outperforms feature fusion in the other two systems. 


Keywords: Multimodal biometric; feature level fusion; 
Matching score fusion; PSO; irsi; palmprint; finger-knuckle 


I. INTRODUCTION 


The unimodal biometric systems are faced with a variety of 
problems, noise in sensed data, non-universality, inter-class 
similarities, and spoof attacks. Multibiometrics overcomes 
these problems besides enhancing matching accuracy. The 
multibiometric systems have many advantages over traditional 
unibiometric systems. They address the issue of non- 
universality. It becomes increasingly difficult (Gf not 
impossible) for an impostor to spoof multiple biometric traits 
of an individual. A multibiometric system may also be viewed 
as a fault tolerant system [1]. 


Multibiometric systems are categorized into three system 
architectures according to the strategies used for information 
fusion [2]: 


Fusion at the feature extraction level: the information 
extracted from the different sensors are encoded into a 
joint feature vector, which is then compared to an 
enrollment template (which itself is a joint feature vector 
stored in a database) and assigned a matching score as in 
a single biometric system. 


Fusion at the matching scores level: feature vectors are 
created independently for each sensor and then compared 
to the enrollment templates, which are stored separately 
for each biometric trait. Based on the proximity of feature 
vector and template, each subsystem now computes its 
own matching score. These individual scores are finally 
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combined into a total score, which is handed over to the 
decision module. 


Fusion at the decision level: a separate authentication 
decision is made for each biometric trait. These decisions 
are then combined into a final vote. 


Fusion at the feature level is a challenging task due to a 
variety of reasons. Most feature sets gathered from multiple 
modalities may be incompatible. Moreover, concatenating 
several feature vectors may lead to construct a relatively large 
feature vector. This definitely increases the computational and 
storage resources demands and eventually requires more 
complex classifier design to operate on the concatenated data 
set at the feature level space [3]. 


The binary particle swarm optimization (PSO) algorithm 
proposed in [4] is applied to perform feature selection. 
Certainly PSO based feature selection has been shown to be 
very efficient in optimizing the feature selection process in 
large scale application problems [5]. PSO also used in other 
fusion levels like matching score level. The PSO is used to 
optimize the selection of score level combination rules, its 
corresponding parameters, and the decision threshold. 


II. RELATED WORKS 


Farmanbar & Toygar [6] employed a method that made use of 
various fusion schemes which were feature-level based also of 
match score level fusion in order to enable proving a robust 
system of recognition. They used face and palm-print systems 
of. A suitable and optimal subset which used palm- print and 
face extracted features was used for the purpose of selecting 
features by using backtracking search algorithm. This reduces 
a computation time minimizing of feature dimension 
meanwhile achieving a higher performance level. A 
performance of match score-level fusion was completed by 
making use of the sum rule method. The results of the 
experiment were later tested on a virtual multimodal database 
by a combination of PolyU palmprint databases and FERET 
face. A noticeable improvement was the result of the proposed 
approach and it outperformed to a significant level other 
palmprint and face multimodal systems that were used in the 
past with a level of 99.17% of accuracy of recognition. This 
method is rightly compared to other state-of-the-art methods. 


Another multimodal biometric system for verification was 
proposed by Motamed et al [7] which was based on two 
different features, the ear and palm. They used a feature 
selection algorithms which was PSO based. Discrete Wavelet 
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Transform (DWT) and Discrete Cosine Transforms (DCT) 
were used in this as its two main features. The process of 
identification for this system was grouped into three phases: 
capturing of image followed by pre-processing which was 
followed by the extraction and normalization of the images of 
the ear and the palm. Then the features were extracted 
followed by matching fusion. The final step was to make a 
decision based on both the GA and the PSO classifiers. The 
results of this experiment showed that this PSO based 
algorithm for feature selection was able to generate a high 
degree of accurate recognition results by making use of 
minimum features that were selected. 


Raghavendra & Dorizzi [8] further presented an effective 
feature scheme of selection with the authentication of 
biometrics for both multimodal and unimodal systems. AIPSO 
(Adaptive Inertia Particle Swarm Optimisation) was thereby 
used for the selection of features of Log Gabor for both the 
face as well as palmprint modalities both independently and 
also on fused Log Gabor space of both these modalities for 
fusion. In case of the final classification for the two schemes 
the space of projection for the features that are selected with 
the use of Kernel Direct Discriminant Analysis (KDDA) is 
considered. The results of the experiment show that the 
performance of AIPSO in comparison to other techniques 
likes Adaptive Boosting (AdaBoost), Sequential Floating 
Forward Selection (SFFS), Normal PSO, Genetic Algorithm 
(GA) an improvement of 5% and a reduction of 62% 
compared to the system that was used initially. 


Kanhangad et al. [9] have presented a promising approach to 
the adaptive management of multimodal biometrics to 
adaptively ensure the desired performance. The authors have 
proposed an algorithm based on Particle Swarm Optimization 
(PSO) to optimally combine the individual biometric sensor 
decisions. The proposed algorithm selected the fusion rule and 
sensor operating points that minimize a given cost function. 


Kumar et al. [10] have introduced an adaptive combination 
system of multiple biometrics to ensure the optimal 
performance for the desired level of security using PSO. They 
have used different biometric combinations (iris, palmprint), 
(face , speech) and (fingerprint , hand geometry). The 
experimental results showed that the proposed score-level 
approach generated fairly stable performance and required 
smaller number of iterations to generate better performance as 
compared to the decision level approach. 


L. Mezai, F. Hachouf [11] proposed an adaptive multimodal 
biometric fusion algorithm based on belief functions and PSO. 
The fusion is performed at the score level, a hybrid PSO is 
employed to select the best belief function and estimate its 
parameters. The results provide adequate motivation towards 
future research in the application of optimization techniques 
in the belief functions. 


Vijaykumar N, Irfan Ahmed M [12] further presented a 
feature selection algorithm also based on Particle Swarm 
Optimization (PSO). They used also Discrete Cosine 
Transform (DCT), PSO, Gabor Filter, correlation based 
selection of feature selection, feature fusion, PSO based 
feature selection as well as classification algorithms like that 
of K-nearest Neighbor and Naive Bayes. Results have proved 
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that this fusion of Naive Bayes was able to improve the rate of 
recognition by 4.3% and 1.81% than when with the DCT-NB 
and Gabor-NB respectively. Likewise, the fusion that was 
proposed with KNN brought a great improvement in the rate 
of recognition by 3.05% and 2.42% than with that of 
DCTKNN and Gabor-KNN respectively. 


Pedro H. Silva et al. [13] have recently propose a deep 
transfer learning optimized from a model trained for face 
recognition achieving outstanding representation for only iris 
modality. They have the proposed iris fine-tuned 
representation and a periocular one from their previous work. 
They compare this approach for fusion in feature level against 
three basic function rules for matching at score level: sum, 
multi, and min. Results are reported for iris and periocular 
region (NICE.II competition database) and also in an open- 
world scenario. The experiments in the NICE.II competition 
databases showed that transfer learning representation for iris 
modality achieved a new state-of-the-art, 1.e., decidability of 
2.22 and 14.56% of EER. They also yielded a new state-of- 
the-art result when the fusion at feature level by PSO is done 
on periocular and iris modalities, i.e., decidability of 3.45 and 
5.55% of EER. 


Hl. APPROACH 


In this paper, we present the proposed method aiming a 
comparison between two multimodal biometric fusion 
systems using different combinations of iris, palmprint and 
finger-knuckle based on feature and score level fusion. 


1) Feature level: Usually, the fused feature vector is 
large in terms of dimensionality and may contain irrelevant or 
redundant information. Moreover, large feature vector also 
increase the storage cost and the consumed time in 
classification. From this point, the feature selection gains its 
absolute necessity in reducing execution time and improving 
recognition accuracy. We proposed a two proposed scenarios 
for the optimized Feature level fusion using (PSO). 


In scheme 1, the features are extracted from each biometric 
iris, palmprint and finger-knuckle separately. The feature 
vectors then fused together. Finally the PSO was applied to 
the fused feature vector to select the most significant features. 
But as the fused feature values of vectors may exhibit 
significant variations both in their range and distribution, 
feature vector normalization is carried out. The objective 
behind feature normalization (also called range-normalization) 
is to modify the location (mean) and scale (variance) of the 
features values and to independently normalize each feature 
component to the range between 0 and 1 [24]. 
Si-— wu 


ð 


S, ' 


1 


d) 


Where: 


S,' isthe normalized matching scores 
S, is the vector to be normalized, and 1 is the no of classes 


u and 6 are the mean and the variance of the fused feature respectively. 
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In scheme 2, the features are extracted from each biometric 
iris, palmprint and finger-knuckle separately. PSO then used 
to select optimized features from each biometric separately. 
The optimized feature vectors then normalized and fused 
together. 


2) Score level: Matching score level fusion is the most 
popular fusion method in the literature. Figure 3 shows the 
block diagram of the proposed system for optimized matching 
scores level fusion using Particle Swarm Optimization (PSO). 
The feature vectors are extracted from each biometric 
separately. Then the matching score for each biometric sample 
is calculated according to the corresponding templates. 
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Fig. 1 The proposed optimized score level fusion 


IV. 
A. 


The PSO algorithm was developed by Kennedy and Eberhart 
in 1995 [14]. The main idea of PSO is inspired from the social 
behavior of organisms, such as birds in a flock. The PSO 
algorithm imitates the behavior of flying birds and their means 
of information exchange to solve optimization problems. Each 
particle (representing a bird in the flock), characterized by its 
position and velocity, represents the possible solution in 
search space. Behavior of the particles in the PSO imitates the 
way in which birds communicate with each other, while 
flying. During this communication, each bird reviews its new 
position in the space with respect to the best position it has 


FEATURE SELECTION USING PSO 


Particle Swarm Optimization (PSO) 
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covered so far. The birds in the flock also identify the bird that 
has reached the best position/environment. Upon knowing this 
information, others in the flock update their velocity (that 
depends on a bird’s local best position as well as the position 
of the best bird in the flock) and fly towards the best bird. The 
process of regular communication and updating the velocity 
repeats until reaching a favorable position. 


Finger 
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Feature 
Selection using 
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Feature 
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Feature 
Selection using 
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Fused Feature 
Vector 


Recognition 


Fig. 2 The proposed optimized feature level fusion 


In a similar manner, the particle in the PSO moves to a new 
position in the multidimensional solution space depending 
upon the particle’s best position (also referred to as local best 
position (Pak) and global best position (Pex). The Pak and Pex 
are updated after each iteration whenever a suitable solution is 
located by the particle (lower cost). The velocity vector of 
each particle represents/determines the forthcoming motion 
details. The velocity updates equation of a particle of the PSO, 
for instance (t+1), can be represented as follows [15]: 


vew = wvf + c ri(pbestpa — xoi 
+ €o%(gbestyg — x (2) 
Where 


œ is the inertia weight between 0-1 and provide a balance 
between global and local search abilities of the algorithm. The 
accelerator coefficients cı and c2 are positive constants, and rı 
and r2 are two random numbers in 0-1 range. 
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The corresponding position vector is updated by: 


old new 


new _ 
= Xnd T Vod 


Xnd 


(3) 


Equation (2) indicates that the new velocity of a particle in 
each of its dimensions depends on the previous velocity and 
the distances from the previously observed best solutions 
(positions of the particle). 


B. Binary PSO 


PSO was initially developed for a space of continuous values 
and it consequently, faced several problems for spaces of 
discrete values. Kennedy and Eberhart [16] presented a 
discrete binary version of PSO method (BPSO) for discrete 
optimization problems. 


In BPSO, particles use binary string to represent thier position 
in form by Xp = {Xp1,Xp2,..., Xpa} which is randomly generated. 
As each bit in the string represents a feature, value =1 means 
that the corresponding feature is selected while =0 means that 
it is not selected. The velocity of each particle is represented 
by Vp = {Vp1,Vp2,---.Vpa}_ , Where p is the number of particles, 
and d is the number of features of a given dataset. The initial 
velocities in particles are probabilities constrained to the 
interval [0.0-1.0]. Each particle is updated according to the 
following equations [16]: 


1 


S (va = ——rew (4) 
l ue 1te “pd ) 
xnew = (1 if (r < S(vpe” ) (5) 
0 otherwise 


Where v,q" denotes the particle velocity obtained from 
new 


equation 2, function S(v,q 


new 
Xnd 


selected from a uniform distribution U (0, 1). 


C. 


) is a sigmoid transformation, 
is the new particle position and r is a random number 


Fitness function 


The PSO implementation relies on the appropriate formulation 
of the fitness function. In the proposed work, a multi objective 
fitness function has been used. The main objectives of the 
fitness function are 


e Maximize the between-class scatter among the different 


classes. 


e Minimize the within-class scatter in the same class. 


e Improve the recognition rate of the system. 


Suppose there are C classes, yi is the i" vector, Mj the number 
of samples within class 1,1= 1,2, ..... C. uithe mean vector of 
class I, and u be the total mean vector of samples. 


Within-class scatter matrix is represented by equation (6) 


Sw = DY Yi — W)C Y: — m) 


i=1 j=1 


(6) 
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Between--class scatter matrix is represented by equation (7) 


-n x He — WC wD 
i (7) 
Where 


C 
u= t/c > H 
i=1 


Finally, we compute a transformation that maximizes the 
between-class scatter while minimizing the within-class 
scatter and this is performed by: 


det (S;) 


Where det () is the determinant of the matrix. 


V. SCORE LEVEL FUSION USING PSO 


Score level fusion refers to the combination of matching 
scores provided by the unimodal classifiers in the system. 
This is the most widely used fusion approach, as evidenced by 
the experts in the field. But before the fusion step, these 
matching score should be normalized. In this paper Min-max 
method is applied which transforms scores into a common 
range [0, 1]. The normalized scores are given by [17]: 
oe Si — S in 
S max — S min (8) 

Where 


S : : the normalized matching scores 


: the matching scores, 


S i 
i= 1,2, .....,n and n: number of matching scores 


S 


min & max ;: the min and max match scores 


In order to combine the scores reported by the three matchers, 
different score level combinations could be applied, such as 
sum, product, weighted sum rule and min rules: 


Sum = y's, 
i=l 


(9) 
Product = Il S, 
i=l (10) 
Weighted _ Sum = Š w, S, 
i (11) 
Where: 
N : number of match scores wanted to be fused 
S : the matching score 
W, : The weight for each score which calculated as follow 


(12) 
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TABLE I. COMPARISON OF UNIMODAL BIOMETRIC RESULTS 


C ampin | 96:76 | 000 | 324 | 324 
[Finger Knuckle | 8550 | 0.00 | 1450 | 1450 


Where EER, is the unimodal biometric error. 





m: the number of biometrics. 


VI. EXPERIMENTAL RESULTS AND DISCUSSION 


Generally, the performance of any biometric recognition 
system is measured by False Acceptance Rate (FAR) and 
False Rejection Rate (FRR) or Genuine Acceptance Rate 
(GAR). The system should have a high GAR with a 
corrosponding low FAR, FRR and Total Error Rate (TER) 
[18]. 


FRR, FAR, GAR and TER are determined as follow: 


false accep tance numbers 


FAR(%) = X 100% 
No of imposter test (13) 
FRR SE i 
No of client test (14) 
GAR(%) = 100— FRR(%) (15) 
TER(%) = FRR(%) + FAR(%) (16) 


A. Unimodal Experimental Results 


For iris images, CASIA iris Image Database is used [19], 
includes 2500 iris images from 250 eyes for each eye. 200 
persons have been selected, for each person 6 Iris images are 
used for training and 4 for testing. 


For palmprint images, PolyU palmprint database is used [20], 
contains 7752 grayscale images corresponding to 386 
different palms (10 samples for each hand). 200 persons have 
been selected, for each person we have 6 palmprint images for 
training and 4 for testing. 


For finger-knuckle images, database images introduced in 
[21] is used, collected from 165 volunteers (12 samples for 
each user), including 125 males and 40 females. 200 persons 
have been selected, for each person 8 finger-knuckle images 
for training and 4 for testing. 


Table 1 shows the results of iris, palmprint and finger-knuckle 
identification systems. It could be noticed that the TER is too 
much to be suitable for high security applications. 


B. Fusion Experimental Results 


The goal of this experiment is to compare the system 
performance when using a feature fusion multimodal 
biometric system by the aid of PSO as an optimizer and a 
matching score fusion multimodal biometric system. 


Table II shows the results of the classification rate including 
FAR, FRR, TER and GAR for the proposed multimodal 
biometric feature fusion approach (scheme 1), feature fusion 
approach (scheme 2) and the proposed multimodal biometric 
score fusion approach by the aid of PSO as an optimizer. 


The results shown that feature fusion system approach 
(scheme 1) achieve significant results with best GAR 98.83 
and TER 1.16%. While the feature fusion system approach 
(scheme 2) achieves significant results with best GAR 98.58% 
and TER 1.41%. Finally, The proposed score fusion system 
achieves significant results with best GAR 98.40% and TER 
2.60%. 


It’s clear that the results of feature fusion scheme 1 
outperform that of scheme 2 in terms of recognition rates and 
total equal error rates. But scheme 2 achieves better results in 
only one case (palmprint_iris). This is because here the 
recognition rate and error basically depends on the rates of 
each biometric separately. But It is clear that the performance 
of the matching score fusion multimodal biometric system 
outperforms the two schemes of feature fusion systems and 
strongly reduces the TER. The proposed system achieves 
significant results with best GAR 98.40% and TER 2%. 


As shown in Table III, we surpass the result presented by M. 
Madane [22] and S. Singh [23] both using score level fusion. 
We couldn't make a comparison with systems using feature 
fusion level and the same biometric traits. 


TABLE I Recognition rates for proposed multimodal systems using PSO 


Biometric System Feature Fusion Scheme 1 Feature Fusion Scheme 2 Matching Score Fusion 


Iris_knuckle | 97.83 83 2: 2e 2. | 216 97.12 | 12 2: | 2.83 2: 283 


Ta a 
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TABLE III Results comparing with literature methods 


es maas O | puamase | rasiona | cana 
O ioone | meram O | | Sereno | oss 
Meats 016% | hi Pampie | UT Dani | sooretuion | 7658 
os 
<a 
E 
os 


Proposed method 
S. Singh (2015) °°) 


A.Meraoumial (2011) °* 


Proposed method Finger-knuckle, Iris PolyU, CASIA 


Proposed method 


VII. CONCLUSION 


In this paper two levels of fusion (feature level fusion and 
matching score level fusion) based on Particle Swarm 
Optimization approach (PSO) are experimented in multimodal 
biometric identification systems with Iris, Palmprint and 
Finger-knuckle biometric traits. 


From the results obtained we can conclude that superior any 
fusion system of them on another depend on number and type 
of biometrics used. 


Although fusion at the feature level is a challenging task due 
to a variety of reasons, but it foretells promising results 
Outperforms score fusion with more experiments along with 
the PSO (or any optimization algorithm with the same 
principle) to reduce the high dimensional features vectors and 
to enhance the quality of the final features vector. 
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